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Abstract 


The incorporation of experimental test data into the 
optimization process is accomplished through the 
use of Bayesian-validated surrogates. In the surro- 
gate approach, a surrogate for the experiment (e.g., 
a response surface) serves in the optimization pro- 
cess. The validation step of the framework provides 
a qualitative assessment of the surrogate quality, and 
bounds the surrogate-for-experiment error on designs 
“near” surrogate-predicted optimal designs. The util- 
ity of the framework is demonstrated through its ap- 
plication to the experimental selection of the trailing 
edge flap position to achieve a design lift coefficient 
for a three-element airfoil. 


Introduction 


To address the inherent difficulties in examining 
many design points experimentally, a three-element 
airfoil model with internally embedded actuators has 
been developed. 1 * The model (Fig. 1) has a nested 
chord of c = 18 in., a span of b = 36 in., and was de- 
signed for low-speed testing in several local tunnels, 
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Figure 1: Three-element model with internal 
flap actuators. 


including the NASA Langley Research Center 2- by 
4-foot and the Old Dominion University (ODU) 3- by 
4-foot low-speed facilities. The main element chord 
is Cmain = 14.95 in., and the flap and slat chords 
(expressed as a percentage of the nested chord) are 
30 and 14.5 percent, respectively. The flap and slat 
are both deflected to 30° for all tests. Although this 
particular model is suitable only for low Reynolds 
number testing, the techniques developed should be 
applicable to higher Reynolds number testing as well. 

The flap actuators are computer controlled and po- 
sition the flap horizontally and vertically (x and y , 
respectively). The model has been used in the ODU 
tunnel to compile baseline values for lift coefficient 
Ci versus flap gap and overhang at fixed angles of 
attack and slat riggings. A first-order optimizer that 
uses a variant of the method of steepest ascent 2,3 has 
been demonstrated in real time. 4 The capability of 
the computer controller to automatically take data 
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at a prescribed set of (x, y) coordinates makes this 
setup ideal for the surrogate methods described next. 

The Bayesian-validated surrogate framework ap- 
plied in this paper provides a practical means to in- 
corporate experimental data directly into the design 
optimization process. In the surrogate approach to 
optimization, a surrogate (i.e., a simplified model, 
for example a response surface) for the experiment is 
constructed from off-line appeals to the experiment. 
The surrogate is then used in subsequent optimiza- 
tion studies. This approach to optimization can be 
contrasted with on-line (direct insertion) strategies, 
in which appeals to the experiment are embedded di- 
rectly into the optimization process. 

The off-line surrogate approach 5-8 to optimization 
offers several advantages to on-line approaches. First, 
by construction, surrogates are computationally in- 
expensive and are thus easily incorporated into op- 
timization procedures. Additionally, the low com- 
putational requirements create a highly interactive 
and flexible design environment, which allows the de- 
signer to easily pursue and examine multiple design 
points. Second, the number of appeals to the experi- 
ment or simulation is known a priori , which ensures 
that the design can be accomplished without exhaust- 
ing available resources. Third the surrogate approach 
offers a natural means to incorporate data from pre- 
vious runs and/or other sources. 

As regards disadvantages, the primary drawback is 
that in high dimensional design spaces, surrogate con- 
struction is difficult and design localization is poor. A 
second limiting factor in the application of the surro- 
gate approach to experimental tests is the need to 
validate the surrogate at input points chosen ran- 
domly in the design space. This capability, present in 
the experiment central to this work, is not typical of 
most experimental tests. Finally, surrogate-based op- 
timization introduces a new source of error. The sur- 
rogate validation strategy and error norms discussed 
in this paper seek to quantify the discrepancy be- 
tween the surrogate and the experiment by providing 
estimates to the system predictability and optimality. 

In this paper, we first describe the experimental 
model and the testing methods used. Second, we 
present the optimization problem that is central to 
the work. Third, we briefly describe the three steps 
of the baseline surrogate framework (i.e., construc- 
tion/validation, surrogate-based optimization, and a 
posteriori error analysis), summarize the inputs to 
the framework, and then present an overview of the 
more sophisticated surrogate algorithms. Finally, we 
present sample results obtained from the surrogate 
framework for output maximization and multiple- 
target designs, and compare the surrogate approach 



with the direct insertion results reported previously. 4 

Experimental Testing Methods 

An important practical problem encountered in wind- 
tunnel testing of multielement airfoils is the need to 
test a range of configurations to ensure that the op- 
timum is selected. Unfortunately, this testing can 
be prohibitively time consuming if one considers all 
possible variables, such as flap position and deflec- 
tion, slat position and deflection, overall angle of at- 
tack, and Reynolds number. For example, a range of 
flap locations and orientations relative to the main 
element is typically tested. In a cryogenic or pres- 
surized facility, model geometry changes necessitate 
large delays in testing. These delays often result in 
investigators choosing a sparse test matrix and an op- 
timum that is based on only a few points. The ability 
to move the flap under computer control provides a 
unique opportunity to explore the entire range of use- 
ful gap and overhang values (Fig. 2). 

In this experiment, the flap actuators, tunnel flow 
setting, and data acquisition were controlled by a per- 
sonal computer running Lab View 9 software. A pro- 
gram was written to allow any number of flap posi- 
tions (in x and y) to be sampled in any order. Wind 
tunnel power was controlled such that at the begin- 
ning of each test the tunnel was restarted to avoid 
hysteresis effects. 4 The experimental setup allowed 
the user to start the program, which at each loca- 
tion in turn automatically measured the free-stream 
properties, sampled and recorded pressures around 
the centerline of the model, and then calculated lift 
coefficients for the three-element airfoil. This process 
required approximately 2 min. for each data point. 

Two typical pressure distributions are shown in 
Figure 3, where the ordinate is the pressure coeffi- 
cient C p and the abscissa is distance from the leading 
edge expressed as a percent of the nested chord. The 
data for Figure 3(a) represents a point near the peak 
Ci for this configuration, and the plot in Figure 3(b) 
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gap s - 2.17 
o.h. s = -1 .46 
x=14.55 y=.45 


a = 8 ° 

C, = 2.72 
Re = 1x1 0 6 


(a) Fully attached flow over all elements. 



gap s = 2.17 
o.h. s = -1.46 
x=14.75 y=.25 


x/c % 


a = 8 ° 

C, = 2.04 
Re = IxlO 6 


(b) Detached flow over the flap. 
Figure 3: Experimental pressure data. 


indicates full separation over the flap. 

Test matrices were developed to survey flap po- 
sitions, which ranged from approximately 0.8 - 3.5 
percent (gap) and —0.4 - 3.4 percent (overhang) rel- 
ative to the nested chord c. Two angles of attack 
and two slat geometries were selected. An angle of 
attack a of 8° was chosen as representative of an ap- 
proach value. An a of 14° represented the limit of 
good-quality two-dimensional flow for the ODU tun- 
nel installation without tunnel wall boundary-layer 
control. Two slat settings were chosen: a slat gap of 
3.03 percent with an overhang of 2.46 percent and, 
for a smaller gap setting, a slat gap of 2.17 percent 
with a slat overhang of —1.46 percent 

Positional accuracy was enhanced by requiring that 
the flap move to a reference point above and be- 
hind the desired evaluation points (x re f > x eva i , 
Vref > Vevai ) and then back to the evaluation point. 
This eliminated any effect of backlash in the mechan- 
ical drive-train. Two simple tests provided an indi- 
cation of the inherent collective error due to instru- 
mentation and positioning. The first test involved 
two separate evaluation points; the first point was 


in a region in which the flow was known to be fully 
attached to all elements, and the second point was 
chosen in a region in which flow over the flap was 
fully separated. The positioning program was used 
to move the flap between a reference point and one 
of the evaluation points. The tunnel was restarted 
before every evaluation, and the test was repeated 30 
times in each case. The standard deviation of Ci was 
found to be 0.004 for the separated case (0.16 per- 
cent) and 0.0118 for the attached case (.36 percent). 
For the second test, the program automatically sam- 
pled 29 points over the entire test region for two dif- 
ferent trials. The error in Ci between the two runs 
averaged 0.71 percent with a standard deviation of 
0.75 percent. Although these tests are not exhaus- 
tive, they do provide a benchmark for the Ci error. 

The turbulence intensity in the ODU tunnel was 
measured at less than 0.2 percent. Flow quality over 
the model was monitored through 12 spanwise taps: 6 
on the flap, and 6 on the main element. The flow was 
considered to be two-dimensional if the magnitude of 
the spanwise nonuniformity was less than 5 percent 
of the total C p variation over the entire model. 10 The 
data presented are uncorrected for boundary effects 
were taken at a Reynolds Re number of 1 x 10 6 based 
on the nested chord. 

Optimization Problem 


We begin by introducing a vector p of M design 
inputs that lie in the input (or “design”) domain 
ft C 1R M , an input-output function <S(p) : ft — >• 1R, 
and an objective function 4/(<S(p),p, A) that charac- 
terizes our design goals, where A is a vector (or possi- 
bly scalar) design parameter. For the work presented 
here, we set p = (x, y) (the x- and ^/-positions of the 
flap) as the M — 2 inputs and restrict ourselves to 
an input domain ft of reasonable flap positions (de- 
scribed in more detail in the results section). The out- 
put of interest is the lift coefficient, <S(p) = Ci(x,y). 
The objective function is 4/(<S(p),p, A) = |<S(p) — A| 
which has been referred to as the “discrimination” 
problem. 11 

With the above terms defined, the minimizer p* = 
(x*,y*) to the exact optimization problem is given by 

p* = argmin|<S(p) - A| . (1) 

In this formulation, the goal is to find that (or “an”) 
input vector p* = (x*,y*) that achieves as closely as 
possible the target lift coefficient value A. If the tar- 
get lift coefficient A is set sufficiently small (large) , the 
formulation describes the output minimization (max- 
imization) problem, assuming that <S(p) is bounded 
from below (above). 
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In the on-line approach, the experiment is invoked 
at every optimization step needed to solve Equation 
(1). In the off-line approach, a surrogate, <S(p) « 
<S(p), for the experiment is inserted into the opti- 
mization problem. The minimizer, p* = (x*,p*), for 
the resulting, surrogate-based, discrimination prob- 
lem is then given by 

p* = argmin|5(p) - A| . (2) 

Here, the optimization proceeds exactly as it would 
for the on-line approach, but the lift coefficient surro- 
gate <S(p) is invoked instead of the experiment. The 
surrogate problem that corresponds to Equation (1), 
but with a general objective function 4/(<S(p), p, A), 
has been reported by Yesilyurt 12 and Yesilyurt and 
Patera. 8 

Surrogate Framework 

The advantages to pursuing a surrogate-based ap- 
proach to optimization have already been described. 
However, to use a surrogate-based approach with con- 
fidence in a design setting, the issues of predictabil- 
ity and optimality must be addressed. 13 For pre- 
dictability, the concern is with how the actual ex- 
periment performs in the vicinity of the surrogate- 
predicted minimizer p*. If the surrogate-predicted 
minimizer is to be of value, we must be able to bound 
|<S(p') — <S(p*)| for p' “near” p*, and this bound 
must be acceptably small. In the case of optimality, 
the designer requires confidence that the surrogate- 
predicted optimizer p* is near the “exact” optimizer, 
that is, p* « p*. Optimality requires stronger as- 
sumptions in regard to the form of the objective 
function (e.g., quasi- convexity) and is, therefore, dif- 
ficult to determine in real applications. Optimal- 
ity is, however, an important consideration and, al- 
though not addressed further here, has been exam- 
ined elsewhere. 8,12 

The distinguishing attribute of the Bayesian- 
validated surrogate methodology is that a com- 
plete and rigorous validation step is fully inte- 
grated into the a posteriori error analysis of the 
surrogate-predicted design (s). The approach de- 

scribed here is related to probably-approximately- 
correct approaches 14,15 and information-based com- 
plexity theory. 16 The surrogate approach differs, 
however, from the former in that it is truly non- 
parametric (no assumption is made in regard to the 
distribution of p*) and from the latter in that it re- 
quires no regularity estimates for the input-output 
function. 

The surrogate approach is broken into three steps. 
In the first stage, surrogate construction/ validation, 


experimental results and/or prior information are 
used to construct the approximation, <S(p) « <S(p); 
additional queries to the experiment are used to val- 
idate the approximation. In the second step of the 
process, surrogate-based optimization, solutions to 
surrogate optimization problem of Equation (2) are 
obtained. In the third and final step, a posteri- 
ori error analysis, the results of the validation are 
used to analyze the consequences of the surrogate-for- 
simulation substitution. In the following subsections, 
we describe the three steps of the baseline surrogate 
framework, summarize the inputs to the framework, 
and review the more sophisticated surrogate algo- 
rithms. 

Construction /Validation 

We construct the lift coefficient surrogate <S(p) = 
A(X co ) « <S(p) using an approximation scheme, 
A : (1R m ,1R) nco — >• L°°(f2) and a construction sam- 
ple set of input-output pairs 

* co = {(pi,R Pi ),i = l,-",W co }, (3) 

where R Pi = Ci(xi,yi) is a realization of the experi- 
mentally measured lift coefficient for the input flap 
position p i = ( Xi,yi ), and N co is the number of 
input-output pairs in the construction sample. Al- 
though the general surrogate framework can handle 
noisy outputs, 17 the noise contribution is neglected in 
the work presented in this paper. Information from 
prior studies, outside sources, or asymptotic behav- 
ior can also be incorporated into the approximation 
process. It is important to note that the surrogate 
framework makes no assumptions in regard to the 
approximation technique and will accept, and assess, 
any approximation A{X co \ Also, no restriction is 
placed on either N co or the distribution of the con- 
struction sample. 

To proceed with the description of the surrogate 
validation, we first introduce the importance function 
p(p). The importance function serves as a probability 
density function for the selection of the validation 
points: 

[ P(p)dp = 1- (4) 

Jn 

The importance function also leads to the notion of a 
p-measure associated with p(p): for any subdomain 
V C ft, 

M 1 *) = / P(p)dp < 1. (5) 

Jv 

The p-measure of V is simply the weighted relative 
M- volume of V. 
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With the importance function p(p) defined, we 
form the validation sample set 

X va = {(P*, Rpi),i = !,•••, N va } , Pi~p(p) , (6) 


where the input flap positions for the validation 
sample set are drawn randomly according to the prob- 
ability density function p(p). In Equation (6), the ~ 
should be read as “is drawn according to the proba- 
bility density function.” The validation sample size 
N va is given by 


jyvci 


In e 2 

ln(l — £ 1 ) 5 * 


(7) 


and £1 and £2 are the two uncertainty parameters 
described below. The model prediction error U is 
computed from the validation sample set X va as 


U = max 

*>i ex™ 


\Rvi -S(Pj)\ 

s(Pi) 


( 8 ) 


where Tp a denotes the input points of the validation 
sample set and p(p) is a strictly positive, error-scaling 
function described in more detail later. 

The result of the construction/ validation process 
is a probabilistic statement that describes the global 
quality of the surrogate <S(p). The validation state- 
ment can be compactly written as 


Pr (M T ) < £ l} > 1 - £ 2, (9) 

where Pr{event} is the “probability of event ” and 
Y c fHs the uncharacterized region defined as 

X = jp e fi| |<S(p) - <S(p)| > Ug{ p)} . (10) 


The Q targets could represent different target lift co- 
efficients during the flap deployment schedule, or re- 
flect the goals at different flight conditions (e.g., take- 
off and landing). 

A posteriori Error Analysis 

To present the predictability results, we must first 
introduce the notion of a prediction neighborhood. 
We begin with a pseudometric A (a, b) defined for 
all (a, b) G Dx(], which determines a “distance” be- 
tween two input points a and b. Then for any subdo- 
main V C 0 we define the radius of V about a point p 
as rx>(p) = max p / G £> A(p,p'). The prediction neigh- 
borhood located at point p with a p-measure of 2 , 
V(p ,z), is that (or a) region V C 0 of p-measure 
2 that minimizes rx>(p). We assume that p lies in- 
side V(p, z) and that V(p, z\) C V(p, Z 2 ) for z\ < Z 2 - 
We can then show that, with probability greater than 
1— £ 2 , for all q E Q, regions T q C V(p q , £ 1 ) of nonzero 
measure exist such that for all p' G T 9 , 

W)-5(p«)|<e(p«). (12) 

It now remains to bound e(p q ) and make precise the 
extent of T q . 

Several bounds are possible on e(p g ), which we de- 
note the predictability gap. If we wish to bound the 
predictability of each design individually, we find that 

e(p 9 ) < £(pVi), V<?eQ, (13) 

where, for p G 0 and 0 < 2 < 1, 

£(p,z) = Ug(p,z) + 6(p,z), (14) 


The p-measure of the uncharacterized region is 
bounded by £ 1 , and the significance level of the non- 
parametric statistical bound is £ 2 . This result can be 
readily proved 12 with order statistics. 18 

For the simple case of g{ p) = 1, Equation (9) states 
that, with probability greater than or equal to 1 — £ 2 , 
the surrogate error is bounded by U over a region of 
ft of p-measure greater than 1 — £ 1 . Although this 
statement is suggestive, it gives neither an indication 
as to the location of T nor the magnitude of the sur- 
rogate error in T. 


and 

g(p,z)= max fi(p'), (15) 

p'ev( P ,z) 

6(p,z)= max |<S(p')-<S(p)|, (16) 

p'ev(p,z) 

and U is the model prediction error from the valida- 
tion step, Equation (8). 

In addition to the joint estimates to the bound on 
e(p g ), we can also bound the average error over the 
Q target designs. In particular, if we assume that the 
'P{p q i e i) are mutually disjoint, it can be shown that 


Surrogate-Based Optimization 

For the optimization problem, we assume that we are 
given Q target drag coefficient values A g , q E Q = 
{1, . . . , Q}. The goal of the optimization is to find 
the surrogate-predicted flap positions that minimize 
the objective function, 

p9 = p*(A g ) = argmin |5(p) - A g |, Vq e Q. (11) 


1_ 

Q 


Q 


E e (p 9 )^ 

q = i 


max 

PeC Q 




where §_ = {/?i , . . . , f3 Q }, and 


(17) 


L 


C L = {0 e 1R l |0 < ft < 1, l = 1, . . . ,L;E A' = 1}, 

1=1 

(18) 
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is the set of convex L-tuples. The “nonparametric 
average” is relevant to multiple-target designs and 
represents the average, as opposed to the worst-case, 
estimate of the predictability. Also, it is important 
to note that this predictability bound is calculated 
entirely in terms of the inexpensive surrogate, <S(p). 

Finally, for a successful validation (i.e. , p p ( T) < 
£ 1 ), we can bound the expectation of the size of T q 
with respect to the validation sample joint probability 
density. The resulting bound is, \/q E Q, 


E 


(^ip IMT)<ei ) - 


1 + 


1 

In s 2 


+ 


£2 


(l-e 2 r 

(19) 

The expression in Equation (19) bounds the average 
p-measure of the region T g , with respect to £ 1 , for 
many validations. 

Several advantages to bounding the errors only to 
within a finite uncertainty exist. 19 First, we achieve a 
sense of stability in that the estimates apply not only 
to a single point, but to regions T q of nonzero mea- 
sure, assuring that many input points p q exist that 
satisfy the error estimates. Second, for the multiple- 
target case the estimates become sharper because 
there is only a single uncharacterized volume of mea- 
sure £ 1 . Equation (17) is the upper bound for the 
distribution of the single £ 1 -sized uncharacterized re- 
gion among the Q designs. This analysis results in 
a bound on the average error which is less than the 
average of the individual predictability gap bounds 
£(p q > £ i)- Finally, because our predictability analy- 
sis is not premised on any particular set of points, 
the designer has flexibility in the choice of the metric 
A (a, b) (discussed further in the next section). 

As mentioned in the introduction, the primary 
drawback to the surrogate approach is the difficult 
construction and validation of the surrogate in high 
dimensional input spaces. We can easily illustrate 
this point if we consider the uniform importance func- 
tion p(p) and a neighborhood of p-measure £ 1 in the 
input domain ft = [0, 1] M . The neighborhood will 
span at least e^ M in one of the input directions which 
rapidly approaches one as M — >• 00. The loss of local- 
ization as M — >• 00 produces a corresponding loss in 
predictability through £(p, £ 1 ) in Equation (14). In 
certain instances, the surrogate approach can be ef- 
fectively applied to problems with high dimensional 
input spaces. This includes cases in which the in- 
puts are highly correlated (e.g., for shape optimiza- 
tion where highly oscillatory geometries are not likely 
optimizers 20 ) or specialized formulations apply (e.g., 
Pareto formulations 21 ). In general however, the sur- 
rogate approach is restricted to a moderate number 
of design variables. 


Summary of Surrogate Inputs 

To summarize the surrogate framework description, 
and to highlight the flexibility of the environment, we 
note that four inputs to the process are determined 
by the user. These are listed below: 

i. An importance function p(p) : ft — > 

ii. An error-scaling function g(p) : II — > 

iii. Two uncertainty parameters, £ 1 and £ 2 , that sat- 
isfy 0 < £ 1 , £ 2 < 1- 

iv. A pseudometric A (a, b). 

Each input provides the designer with flexibility, and 
allows the designer’s experience to impact and im- 
prove the final surrogate-predicted designs. Although 
poor choices for the inputs do not influence the valid- 
ity of the surrogate results, they greatly reduce the 
sharpness of the results. A short description and ex- 
planation of each input follows. 

The importance function p{ p) reflects the designers 
prejudices in regard to the regions of Vt that are more 
likely to contain optimizers. In this context, p{ p) 
is essentially a “prior” on p*. To serve this purpose, 
p(p) is used as the probability density function in the 
random selection of validation points in Equation (6). 
A judicious choice of p{ p) (one that is large in the re- 
gions of the final designs and small elsewhere) can 
significantly increase the sharpness of the a posteri- 
ori error bounds. The increased sharpness is a conse- 
quence of much better physical localization (in terms 
of input variable extent) of the prediction neighbor- 
hood V(p*,£i), which in turn reduces the surrogate 
sensitivity contribution £(p*,£i) to the error bound 
in Equation (14). 

The error-scaling function g(p) can be used by the 
designer to reduce the impact of localized surrogate 
errors on the error bounds of the final design. Be- 
cause the model prediction error U in Equation (8) is 
global, a large value of g(p) in regions for which the 
approximation is poor will result in a reduced value 
of the first term on the right-hand side of Equation 
(14), provided that the final design does not lie in a 
region where g{ p) is large. 

The uncertainty parameters £1 and £2 are related 
to the number of validation points through Equation 
(7). This formula allows the precise budgeting of re- 
sources and ensures that useful solutions can be ob- 
tained. In effect, Equations (7)-(10) describe what is 
known in a continuous sense about a function based 
on discrete sampling. Analysis of Equation (7) shows 
that, asymptotically for small E\ and £ 2 , N va in- 
creases linearly as £1 decreases and only logarithmi- 
cally as £2 decreases. This relationship suggests that 
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although we can easily (in terms of validation sample 
size) increase our confidence in the results (smaller 
£ 2 ), refining the localization of our results (through 
smaller £ 1 ) is much more difficult. The localization 
has a direct impact on the final error analysis through 
J(p,£i) in Equation (14). The relative difficulty in 
further refining the localization illustrates the need 
to intelligently select p{ p) and where appropriate, 
A (a, b), both of which can have similar effects on 
the localization error. 

The final input to the surrogate approach is the 
pseudometric A (a, b). Because A (a, b) can be chosen 
post- validation, various metrics can be examined, and 
the most appropriate selected. One possible trade- 
off is between design localization (in terms of input 
variable extent) and predictability in terms of 5(p, £ 1 ) 
in Equation (14). An example of the extreme of this 
trade-off is the sensitivity minimizing metric 

A (a, b) = |5(a) — <S(b)| (20) 

used for the single-point design study of the results 
section. This metric gives the lowest possible 5(p, £ 1 ). 

Improved Algorithms 

Several, more sophisticated surrogate algorithms 
have been developed 8,12,17,19-23 but are not de- 
scribed here. First, a surrogate formulation for noisy 
outputs has been developed. 17 This formulation is 
clearly appropriate in an experimental setting but 
is not addressed here. Second, the multiple output 
case can be efficiently handled, and the formulation 
can be applied to model selection. Third, elemen- 
tal decompositions of Vt are possible that yield lo- 
cal errors and allow for rigorous construction/cross- 
validation schemes. 24 Fourth, sequential and adap- 
tive techniques have been developed that allow the 
incremental deployment of resources to achieve tar- 
get surrogate accuracies and that more tightly couple 
the construction and validation phases of the baseline 
algorithm. Finally, nested validation, in which a hi- 
erarchy of models exists (e.g., an extremely expensive 
“truth” model — >• a high-fidelity model — >• low-fidelity 
model), has been addressed as well. 

Results 

To demonstrate the surrogate framework, we have 
applied it to the experimental design of multielement 
airfoils; specifically, we are interested in determin- 
ing the optimal location for the trailing edge flap, 
based on the lift coefficient Ci in low-speed, high-lift 
flight regimes. The M — 2 design inputs to the prob- 
lem p = (x, y) are the x and y positions of the flap, 


measured from the leading edge of the main airfoil 
element and normalized by the main element chord 
Cmain = 14.95 in. The output of interest is (7/. In ad- 
dition, several other configuration and flow condition 
parameters are fixed for the study. These parameters 
are listed in Table 1 and are the Reynolds number Re, 
the airfoil angle of attack a , the flap and slat deflec- 
tion angles Sfi ap and 5 s i a t, respectively, and the gap 
and overhang of the slat (expressed as a percentage 
of the nested chord c = 18.0 in.). 

In this section, we first describe the method used 
for the surrogate construction and report the vali- 
dation results. Second, we consider the single-point 
design problem of output maximization. Third, we 
pursue a multiple-target design study which demon- 
strates the increased sharpness of the nonparametric 
average error results. Finally, we report the results 
of on-line optimization studies and compare these re- 
sults with the off-line, surrogate results. 

Surrogate Construction /Validation 

The construction sample set X co consists of 119 
input-output pairs that are uniformly spaced on a 
17 x 7 grid. The (x,y) flap positions for the con- 
struction sample are plotted as circles in Figure 4. 
The input domain is divided into three subdomains, 
ft = U U based on the flow conditions over 
the flap. In the first subdomain fli, the flow over 
the flap is attached, with the exception of the ex- 
treme aft positions in which some trailing-edge sepa- 
ration may be present (and desirable). In this region, 
a radial basis function 25 serves as the approximation 
method, which yields the surrogate <Si(p). In fl 3 , the 
flow over the flap is fully separated, and a second ra- 
dial basis function fit serves as the surrogate <S 3 (p). 
In fl 2 , the resolution of the construction points is 
not sufficient to determine the precise location of the 
separation line. In this region, a simple linear tri- 
angulation between <Si(p) and <S 3 (p) is used as the 
surrogate, <S 2 (p). The error function, g( p), is set to 
unity in Qi and fl 3 , and g(p) = 50 in fl 2 , reflect- 
ing our uncertainty in regard to the location of the 


Re 

1,000,000 

a 

14° 

Sfiap 

o 

O 

CO 

Sslat 

-30° 

gap slat 

2.17% 

overhang s iat 

-1.46% 


Table 1: Fixed design study parameters. 
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Figure 4: Surrogate construction points and 
the input (“design”) domain. 


separation line and, hence, our lack of confidence in 
the quality of the surrogate in this region of the in- 
put space. A three-dimensional surface plot of the 
surrogate is shown in Figure 5. 

To validate the lift coefficient surrogate, we must 
select a set of random input points in £2 and run the 
experiment at each of these points to form the valida- 
tion sample set X va . The input points are confined 
to the design space £2 described in the previous para- 
graph and shown in Figure 4. Because the construc- 
tion data were obtained simultaneously with the vali- 
dation data, we had no expectation in regard to those 
regions of the input space that would be of most inter- 
est; thus, we used a uniform probability density func- 
tion p{ p) for the selection of the validation points. 
We budgeted N va = 45 points for validation and, us- 
ing the relationship in Equation (7), set e 1 = 0.03 
and £2 = 0.25. If we had known the form of the sur- 
rogate prior to taking the validation data, we could 
have restricted the design space to a more feasible re- 
gion and perhaps chosen an importance function p{ p) 
that would have concentrated validation points close 
to potential designs. The scaled model prediction er- 
ror computed according to Equation (8) is U — .0482. 
Note that the maximum un-scaled error does in fact 
occur in £2 2 as we presupposed and has a value of 
0.4824. If we had chosen g(p) = 1 everywhere (in- 
stead of as described above), our model prediction er- 
ror would have been approximately one order of mag- 
nitude larger, and would surely have overwhelmed the 
results. 

The surrogate just described and the related vali- 
dation results serve for all of the designs discussed in 
the remainder of this paper. One primary advantage 
to using the surrogate approach is the fact that no 
additional experimental data are required to bound 
the errors of future designs that are pursued with 
the surrogate. This characteristic, combined with 


the negligible computational time required for each 
surrogate evaluation, yields a highly flexible design 
environment that does not sacrifice predictability. 

Single-Point Design, Surrogate Maximization 

For the first study, we pursue a single-point design 
that maximizes the surrogate output. We set A suffi- 
ciently large in Equation (2) and minimize the result- 
ing function. To accomplish the optimization, we use 
the unconstrained quasi-Newton optimizer that is in- 
cluded in the optimization toolbox of Matlab 26 . The 
resulting surrogate-based optimizer is located at p* = 
(x*,y*) = (.997, .036), and the surrogate-predicted 
lift coefficient value at this point is <S(p*) = 3.388. 
The optimizer was started with an initial guess at 
Po = (.987, .033) and required 44 surrogate evalua- 
tions to arrive at p*. Because the surrogate is inex- 
pensive to evaluate (and because we are working with 
only two inputs and can visualize the results graphi- 
cally), we can verify that we do achieve a surrogate- 
predicted global maximum. This verification would 
be more difficult in a purely on-line optimization set- 
ting if we did no begin the optimizer at multiple start- 
ing points po until we had sufficient confidence that 
a global maximum had been obtained. 

Finally, we choose the sensitivity minimizing met- 
ric A(a, b) = |<S(a) — 5(b) | in Equation (20) and 
perform the a posteriori error analysis for a single- 
point design. We construct the prediction neighbor- 
hood V(p* , £ 1 ) around p* and find the surrogate sen- 
sitivity parameter S = .0328. The optimal point p* 
and the associated prediction neighborhood V(p*,£i) 
are plotted in Figure 6. The resulting predictabil- 
ity statement reads as follows: with confidence level 
greater than .75, a region T C V(p*,£i) of nonzero 
measure exists such that for all p ' G T 

W)-5(p*)|<e(p*), (21) 

where 

e(p*) < Ug( p*, £ 1 ) + S = .0810 . (22) 

We see that the predictability is relatively good with 
respect to the surrogate-predicted maximum lift co- 
efficient, but quite poor with respect to the range of 
lift coefficients of interest (i.e. , corresponding to flap 
positions in £2i). 

Multiple- Target Designs 

For the second design study, we pursue a multiple- 
target design. The motivation for such a study might 
be an interest in examining the lift coefficient at 
more than one point of the deployment of the flap. 
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<S(p) 



Figure 5: Three-dimensional mesh plot of the lift coefficient surrogate <S(p). 



Figure 6: The surrogate— predicted optimizer, 
p*, and the associated prediction neighbor- 
hood, V(p*,£i). 

Specifically, we want to obtain two target lift coef- 
ficients: A 1 = 3.31 and A 2 = 3.25. Isocontours of 
the surrogate indicate that a locus of points in ft ex- 
ists for each target that exactly satisfies the design 
goals. We arbitrarily select one point for each design: 
p 1 = (#W, 2 /W) = (.987, .033) and p 2 = = 

(.979, .033). Around each optimizer, we construct a 
prediction neighborhood chosen from the family of el- 
lipses that have area equal to e i, are centered at p g , 
and are oriented such that they minimize surrogate 
sensitivity S(p q ,£i). The optimizers and associated 
prediction neighborhoods are plotted in Figure 7. 

For each of the designs (q = 1,2), we can state 
with confidence level greater than .75 that a region 
T q C V(p q ,£i) of nonzero measure exists such that 
for all p' G T q 

W)-«S(p')[<e(p«), (23) 


where 

e(p 1 ) =U + 6{p 1 ,e 1 ) = .0482 + .0198 = .0680, (24) 
and 

e(p 2 ) = U + S( p 2 ,ei) = .0482 + .0201 = .0683. (25) 

The above bounds jointly hold on each design. We 
obtain a slightly sharper bound on the average error 
of the two designs: 

^(p 1 ) + e(p 2 )] < U + .0149 = .0631. (26) 

The increased sharpness results from an analysis of 
the worst-case distribution of the uncharacterized re- 
gion between the two prediction neighborhoods. Be- 
cause of the low sensitivity of the surrogate in each of 
the prediction neighborhoods relative to model pre- 
diction error f7, the improvement is slight. 

Comparison with Direct Insertion 

To date, cases at identical flow conditions have not 
been examined with both on-line (the method of 
steepest ascent) and off-line (the surrogate approach) 
optimization methods. However, rough comparisons 
of the resource requirements are of (guarded) use. 

The on-line results have been reported in an ear- 
lier paper by Landman and Britcher. 4 In that effort, 
they found the optimizer to be very robust (successful 
in 6 out of 6 attempts) and insensitive to the initial 
guess. For each case, they started the optimizer at 
in initial flap position with a low Ci value and ob- 
tained a final value within approximately 0.7 percent 
of the maximum Ci value in approximately 20 opti- 
mizer steps, requiring approximately 60 experimental 
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Figure 7: The surrogate— predicted optimizers 
and the associated prediction neighborhoods 
(shaded). 

data points (3 points per step). With the surrogate 
method, we required 119 points to construct the sur- 
rogate and an additional 45 for the validation, for a 
total of 164 experimental data points. For the max- 
imization problem, the a posteriori error bound was 
2.4 percent of the maximum surrogate value. 

While the surrogate approach seems to compare 
unfavorably to the on-line method, several subtleties 
lie in its favor. First, for designs chosen with the 
validated surrogate in the future (e.g., the multiple- 
target design examined in this paper), similar error 
bounds still apply and do not require additional ex- 
perimental data. In contrast, the on-line approach 
would require additional experimental results. Sec- 
ond, a total of 60 evaluations to obtain an optimal 
point with the on-line method can be deceptive; to be 
assured that the result is indeed optimal, additional 
information is required. The additional information 
for the study cited was in the form of contour plots 
of a matrix of data. If visualization is not possible, 
a number of optimizer restarts would be required to 
be assured of an optimal. Third, in cases for which 
the objective function is less forgiving, restarts of the 
on-line optimizer would be unavoidable, which would 
further increase the required experimental data to a 
level surpassing that of the surrogate approach. Fi- 
nally, the obvious difficulty in pursuing on-line opti- 
mization is related to the ultimate application; if the 
intent is to incorporate the data as a portion of a 
larger optimization study, no alternative is available 
other than to store the experimental data for later 
use and extract with some form of an approximation. 
If one is restricted to a purely experimental setting, 
then the ability to quickly, and automatically, find 
optimal operating points with the on-line optimizer 
is highly advantageous. 
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